Goto

Collaborating Authors

 Chess


Don't be fooled--LLMs don't reason

MIT Technology Review

Ten years after AlphaGo's match against Go champion Lee Sedol, today's AI still isn't tapping into the machinery that made that win possible. On an afternoon in Seoul in March 2016, I watched a program I helped build put a stone on the fifth line of a Go board in what looked like a gift to its human opponent. Move 37 in game two of the five-game match looked so absurd that some commentators thought it was a programming glitch. AlphaGo won the game, ultimately triumphing 4-1 over Lee Sedol, one of the greatest professional Go players of all time. "I thought AlphaGo was based on probability calculation and that it was merely a machine," Lee said afterwards . "But when I saw this move, I changed my mind.


'I was beating adults at chess when I was eight'

BBC News

'I was beating adults at chess when I was eight' When I was eight or nine I was beating adults at chess and they were raging when they lost, like'I just lost to a little kid'. Ethan first became interested in chess when he was six years old after he spotted someone else playing. Now 13, he plays for Strand Chess Club in east Belfast and has represented Ireland through all his age groups. Last week he became the youngest Ulster chess champion in history - not just in his age category but across all ages. I was really surprised because I've never won any of the big tournaments before, I've always got so close and I managed to win it in the end, Ethan told BBC News NI.


The Download: OpenAI's turning point for math and a battery record

MIT Technology Review

Plus: The US has accused six Chinese AI firms of "industrial-scale" theft. What OpenAI's latest controversy tells us about the future of math OpenAI says its agents have solved one of the most important open problems in mathematics. Under normal circumstances, that would be a huge milestone. But the announcement has been overshadowed by accusations that OpenAI failed to credit researchers whose AI-assisted work influenced its solution. Whether those accusations are true or not, the episode may mark a turning point in the history of mathematics. AI models now seem essential for making progress on the field's most important problems, but solving them may demand resources available only to a couple of frontier AI companies.


ChatGPT can now control your whole desktop. I tested it with chess

PCWorld

PCWorld tested ChatGPT's new "computer use" functionality in the recently released super-app, which allows the AI to control desktop applications on macOS. The GPT-5.6 Sol model successfully played chess and wrote in Journal apps, demonstrating advanced understanding of game strategy and graphical interface navigation. This desktop control capability positions ChatGPT alongside Claude's similar features, with OpenAI suggesting uses for testing and automating repetitive tasks. Just a few months ago, I could barely get an AI agent to open the Chess app on my Mac, much less actually play a game of it. But after testing the all-new ChatGPT super-app, I can report that the days of clunky AI "computer use" functionality are pretty much over. Released last Thursday, the ChatGPT super-app boasts a variety of features, including the built-in Codex coding harness and ChatGPT Work, an AI agent that can tap into your local files and create detailed reports in seconds. The revamped ChatGPT also supports "computer use," allowing it to take control your PC's mouse and desktop apps. Computer use functionality isn't new--the Claude desktop app has supported it for months--but it new for ChatGPT, so I decided to give it a go. I fired up ChatGPT on my Mac mini, switched the app to "Work" mode, set the model to GPT-5.6 Sol ( the latest and greatest OpenAI model), and gave it a simple prompt: "Can you play chess on my Mac?" ChatGPT asked me to approve some new permissions, including the ability to record my screen so it could see what it was doing.


Evaluating the Inductive Abilities of Large Language Models: Why Chain-of-Thought Reasoning Sometimes Hurts More Than Helps

Neural Information Processing Systems

Large Language Models (LLMs) have shown remarkable progress across domains, yet their ability to perform inductive reasoning--inferring latent rules from sparse examples--remains limited. It is often assumed that chain-of-thought (CoT) prompting, as used in Large Reasoning Models (LRMs), enhances such reasoning. We investigate this assumption with creating four controlled, diagnostic game-based tasks--chess, Texas Hold'em, dice games, and blackjack--with hidden humandefined rules. We find that CoT reasoning can degrade inductive performance, with LRMs often underperforming their non-reasoning counterparts. To explain this, we present a theoretical framework that reveals how reasoning steps can amplify error through three failure modes: incorrect sub-task decomposition, incorrect sub-task solving, and incorrect final answer summarization. Based on our theoretical and empirical analysis, we introduce structured interventions that adapt CoT generation according to our identified failure types. These interventions improve inductive accuracy without retraining. Our findings suggest that effective (CoT) reasoning depends not only on taking more steps but also on ensuring those steps are well-structured.


miniF2F-Lean Revisited: Reviewing Limitations and Charting a Path Forward

Neural Information Processing Systems

We perform a thorough analysis of the formal and informal statements in the miniF2F benchmark from the perspective of an AI system that is tasked to participate in a math Olympiad consisting of the problems in miniF2F. In such setting, the model has to read and comprehend the problems in natural language, formalize them in Lean language, then proceed with proving the problems, and it will get credit for each problem if the formal proof corresponds to the original informal statement presented to the model. Our evaluation results reveal that the best accuracy of such pipeline can be about 36% using the SoTA models in the literature, considerably lower than the individual SoTA accuracies, 97% and 69% reported in the autoformalization and theorem proving literature. Analyzing the failure modes, we trace back a considerable portion of this drop to discrepancies between the formal and informal statements for more than half of the problems in miniF2F. We proceed with correcting all the errors, discrepancies and simplifications in formal and informal statements, and present the miniF2F-v2 with fully verified formal and informal statements and proofs. Evaluating the full theorem proving pipeline on miniF2F-v2 leads to the best accuracy of 70%, a significant improvement from the 40% on the original miniF2F, yet indicating considerable misalignment between the autoformalization models and theorem provers. Our deep analysis suggests that a higher quality benchmark can help the community better evaluate progress in the field of formal reasoning and also better diagnose the failure and success modes of autoformalization and theorem proving models.


Generating Creative Chess Puzzles

Neural Information Processing Systems

While Generative AI rapidly advances in various domains, generating truly creative, aesthetic, and counter-intuitive outputs remains a challenge. This paper presents an approach to tackle these difficulties in the domain of chess puzzles. We start by benchmarking Generative AI architectures, and then introduce an RL framework with novel rewards based on chess engine search statistics to overcome some of those shortcomings. The rewards are designed to enhance a puzzle's uniqueness, counter-intuitiveness, diversity, and realism. Our RL approach dramatically increases counter-intuitive puzzle generation by 10x, from 0.22% (supervised) to 2.5%, surpassing existing dataset rates (2.1%) and the best Lichess-trained model (0.4%).


Can Large Language Models Master Complex Card Games?

Neural Information Processing Systems

Complex games have long been an important benchmark for testing the progress of artificial intelligence algorithms. AlphaGo, AlphaZero, and MuZero have defeated top human players in Go and Chess, garnering widespread societal attention towards artificial intelligence. Concurrently, large language models (LLMs) have exhibited remarkable capabilities across various tasks, raising the question of whether LLMs can achieve similar success in complex games. In this paper, we explore the potential of LLMs in mastering complex card games. We systematically assess the learning capabilities of LLMs across eight diverse card games, evaluating the impact of fine-tuning on high-quality gameplay data, and examining the models' ability to retain general capabilities while mastering these games. Our findings indicate that: (1) LLMs can approach the performance of strong game AIs through supervised fine-tuning on high-quality data, (2) LLMs can achieve a certain level of proficiency in multiple complex card games simultaneously, with performance augmentation for games with similar rules and conflicts for dissimilar ones, and (3) LLMs experience a decline in general capabilities when mastering complex games, but this decline can be mitigated by integrating a certain amount of general instruction data. The evaluation results demonstrate strong learning ability and versatility of LLMs. The code is available at https://github.com/THUDM/


Generating Creative Chess Puzzles

Neural Information Processing Systems

While Generative AI rapidly advances in various domains, generating truly creative, aesthetic, and counter-intuitive outputs remains a challenge. This paper presents an approach to tackle these difficulties in the domain of chess puzzles. We start by benchmarking Generative AI architectures, and then introduce an RL framework with novel rewards based on chess engine search statistics to overcome some of those shortcomings. The rewards are designed to enhance a puzzle's uniqueness, counter-intuitiveness, diversity, and realism. Our RL approach dramatically increases counter-intuitive puzzle generation by 10x, from 0.22\% (supervised) to 2.5\%, surpassing existing dataset rates (2.1\%) and the best Lichess-trained model (0.4\%). Our puzzles meet novelty and diversity benchmarks, retain aesthetic themes, and are rated by human experts as more creative, enjoyable, and counter-intuitive than composed book puzzles, even approaching classic compositions. Our final outcome is a curated booklet of these novel AI-generated puzzles, which is acknowledged for creativity by three world-renowned experts.


Best Smart Chess Boards (2026): Chessnut, Millennium

WIRED

I played the ultimate game of strategy on a variety of smart chess boards to find the best for online and in-person matches. Playing chess can be challenging, fun, and at times frustrating. Garry Kasparov called the game "mental torture." With virtually limitless possibilities, chess offers unparalleled depth, and you could easily fill a library with books on how to play it. The internet has opened up a wealth of potential competitors, and smart chess boards enable you to play anyone online or off, not to mention dabble in a variety of chess programs.